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ABSTRACT 



This study evaluated the impact of a cluster program and the 
First Year Experience Seminar (First YES) on a variety of student outcomes at 
Boise State University in Idaho. To evaluate the cluster program, which 
included 57 students, a control group of 102 students was formed of similar 
students who were enrolled in the same core courses but not taking them 
together as a group. Two groups were formed to compare First YES enrollees : 
one consisted of students in general education (GE) courses while the second 
consisted of students enrolled in neither First YES nor other GE courses. The 
study found that cluster students were more likely to re-enroll in the spring 
semester and again the following fall compared to the control group. First 
YES and other GE students were also more likely to re-enroll in the spring 
than the control group students. After adjusting for entering academic 
ability, other GE students had the highest first term grade point averages 
(GPAs) , followed by First YES students and the control group students. An 
appendix contains a copy of the student survey. (MDM) 
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AbsTRACT 

An EvAluAiioN of tIhe EarIy Impacts of tIhe CIuster 
PR oqRAM ANd First Year Experience SemInar on New FresIhmen 

This report evaluates the impact of the cluster program and First Year Experience Seminar (GE 
197) on a variety of student outcomes including retention, grade point average, use of and 
satisfaction with Boise State University services, perceived impact of the University on skills 
development, social experiences, and general perceptions of the University. To evaluate the 
Cluster program, a control group was formed of similar students who were enrolled in the same 
core courses but not taking them together as a group. Two groups were formed to compare to 
First YES enrollees. One consisted of students enrolled in other GE courses, mostly GE 108. 
The other consisted of the remaining students who enrolled in neither. The effects of admissions 
Index, age, and number of credits attempted first semester were statistically controlled to make 
the groups as equivalent as possible. 

The following is a summary of major findings: 

• Cluster students were more likely to re-enroll in the spring and again the following fall 

compared to the control group. In the spring, 91% of cluster students returned compared to 
78% of controls. In the Fall, 68% returned compared to 52% of controls. No differences 
were found for any other variables, including GPA, use of and satisfaction with University 
services, impact on skills development, or social experiences. 



• First YES and Other GE students also were more likely to re-enroll in the spring than the 
controls (90% vs. 80%). The differences did not hold for the following fall term, however, 
except for part-time students. 

• After adjusting for entering academic ability. Other GE enrollees had the highest first term 
GPAs. They were followed by First YES enrollees, while the control group had the lowest 
GPAs. Results were not as strong for spring term GPAs. 



• Compared to the control group, both First YES and Other GE enrollees thought the 

University had impacted them more highly. In addition, they had used more services and had 
participated in more study groups. 

. Compared to the control group. First YES students were more likely to work with other 
students on projects, express satisfaction with services, and generally have better perceptions 
about their first semester at Boise State. This was not true for students enrolled in the other 

GE courses. 

These findings indicate that early retention efforts seem to be paying off. Because the First YES 
(GE 197 ) and Other GE (mainly GE 108 ) students had many similar results despite initial 
differences in the kind of students enrolled in the two courses, a variety of orientation and study 
skills courses should probably continue to be provided. 



An EvaIuatIon of iliE EarIy Impacts of tIte CIusier 
PR oqRAM ANd First Year Experience Course on New FresItmen 

What to do about retention at Boise State University? That was the general issue facing the 
Retention Committee. After research into what was working at other institutions and gathering 
local data, the committee issued a report in 1993 with 13 recommendations. One 
recommendation was to implement a cluster program that grouped courses together so that 
students enrolled in the cluster would have four classes with the same group of students, thereby 
allowing for the establishment of a sense of community — to make friends and develop a support 
group within the larger, more impersonal university setting. Another was to develop a one-credit 
course designed to help freshman and transfer students make successful transitions to the 
University and to assist them in reaching their educational goals. The resulting course was 
designed to provide students with academic skills training and essential information about the 
University, its rules, procedures and resources. Students also learned strategies for making 

appropriate major and career path choices. Named “First Year Experience Seminar or First 

YES, all students enrolled in the cluster program took the course. Other new enrollees at the 
University were also strongly encouraged to enroll. Both new efforts kicked off in the Fall of 
1995. This study reports on the initial outcomes from the program. 

Outcomes StucHecI 

Above all else, it was hoped that the efforts would improve retention. This, after all, was the 
purpose of implementation. It would also be useful, however, to see if the program had impacts 
in some other areas that were either directly or indirectly related to retention. Toward this end, 
the following outcomes were included in the study: 

Retention: Defined as re-enrolling the next term and the following fall. These were two critical 
periods for losing new freshmen. 

Grade Point Average (GPA): Defined as first and second semester term GPAs. Students with 
lower grade point averages, especially below 2.0, have been found to be less likely to return to 
Boise State University (see Research Report 97-01). 



Use of Sen’ices: Defined as the number of services students indicated they had used at the end of 
their first semester. Since First YES included familiarizing students with University resources, 
this seemed a viable outcome to study. A listing of the 16 services included can be found in the 
survey contained in Appendix A (items 1 5-30). 

Satisfaction with Scnnccs'. Defined as the sum of the satisfaction ratings students gave for those 
services they used. It was theorized that greater understanding of services and what they had to 
offer might lead to greater satisfaction. 

Impact of Boise State University on skills development: Defined as the sum of the impact 
ratings in eight areas: commitment to lifelong learning, solving problems, developing job skills, 
learning about careers, getting along with different people, using effective oral and written 
communication, and drawing conclusions from data. The items used can be found in Appendix A 
(items 1-8). 

Social Experiences: Students indicated how frequently they had held a conversation with faculty 
members and other students, worked with others outside of class on a project, and met as a 
member of a study group. See Appendix A, items 31-34 for details. 

General Perceptions of Boise State University: Defined by five items asking about Boise State 
University’s environment, teaching, and helpfulness of staff. (See Appendix A, items 9-13.) 

Comparison Groups 

Students enrolled in the cluster program were all full-time students (9 or more credits) and, except 
for one, under the age of 20. Therefore, the cluster comparison group was also limited to full- 
time enrollees under the age of 20. In addition, every student in the comparison group took the 
same set of courses as students in the cluster program; they simply were in different sections so 
should not have seen the same students in all their classes. To ensure that gender and admissions 
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index scores did not differentially influence the outcomes for the two groups, these two variables 
were assessed as part of the statistical analysis and any differences were statistically controlled. 

Two comparison groups were formed for the First YES (GE 197) group. One consisted of 
students enrolled in other skills courses (mainly GE 108, but also GE 100, GE 114, and GE 1 15). 
Most students (N=106) were enrolled in GE 108, a two-credit reading and study skills course 
designed to assist students in meeting the demands of their university courses. A significant 
number (N=46) were enrolled in GE 100, a two-credit course title Strategies for Academic 
Success”. Only two students were enrolled in GE 1 14, a “re-entry” course for older students, and 
seven were enrolled in GE 115, Career and Life Planning. The second group included students 
enrolled in none of the GE courses. Because of the diversity of the First YES group, a number of 
additional variables were included to see if they had different effects for the First YES group 
compared to the others. Besides admissions index scores and gender, checks were made to see if 
age and number of attempted credits needed to be statistically controlled in the analysis. 

SiATisricAl ANAlysis 

For all variables except re-enrollment, regression analysis using the General Linear Model (GLM) 
was employed. In each case, the analysis commenced with checking to see if the covariates (i.e., 
admissions index, credits, age) had a significant impact on the analysis. If not, they were 
eliminated from further study. If they did, the next analysis looked to see if the covariates had 
different effects depending upon the comparison group being studied. Then differences between 
the comparison groups were tested for statistical significance after taking into account the effects 
of the covariates. Gender was included in the model as a main effect but eliminated from further 
analysis if an interaction with the comparison groups was not found. When necessary, a one-way 
Analysis of Variance and Tukey’s for post hoc comparisons were used. An alpha level of .05 was 
used to establish statistical significance. For re-enrollment, chi-square testing was used. 

XUe HfECTS of jUe CluSIER PROqRAM 

Results revealed that significantly more cluster students returned next semester and the following 



fall compared to control group students. Table 1 shows that at least 13% more cluster students 
returned in the Spring and 16% more in the Fall than control group students. 



Table 1 

Re-Enrollment Patterns for Fall 1995 
Cluster Students and Controls 



Group 


N in Group 


% Returning 
Spring 1996 


% Returning, 
Fall! 996- 


Control 


102 


78.4 


52.0 


Cluster 


57 


91.2 


68.4 


NOTE: Statistical signi 


ficance was found for both terms 



For the remaining variables including GPA, social interactions, use of and satisfaction with 
services and perceptions of BSU, no significant differences could be found between the means for 
the cluster group and the control group (see Table 2). This included first and second semester 
GPA as well as measures of service use and satisfaction, social interactions, and general 
impressions of Boise State University. Note that very few people in the control group completed 
the survey so this may have effected the possibility of finding statistical significance. 

Effects of First YES Course (GE 197) 

Again, a relationship was found between enrolling in study skills courses and returning in the 
Spring. About 90% of students who enrolled in either First YES or another GE courses returned 
for the spring semester compared to about 80% of those who did not. Spring term differences 
were found for all groups, but enrolling in First YES appeared particularly helpful for part-time 
students and women. The relationship did not hold for the following Fall term, however, except 
for part-time students. See Table 3 for further details. 

Enrolling in student success courses also was related to higher first and second term grade point 
averages. Table 4 shows the first and second term GPAs for the groups after they had been 
adjusted to account for the differences in admissions index scores. For the first semester, all 
groups differed significantly from one another with the Other GE group showing the highest 
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Table 2 

Means for Cluster and Control Groups 





Control Group 


Ouster Group 


Variable 


N 


Mean 


i;'-- N 


Mean 


First Term GPA* 


102 


2.34 


57 


2.49 


Second Term GPA* 


93 


2.27 


51 


2.25 


BSU’s Impact 


8 


18.13 


50 


16.26 


Faculty Conversations 


7 


3.86 


49 


3.22 


Student Conversations 


7 


5.00 


49 


4.96 


Project work 


7 


2.43 


49 


3.35 


Study Group 


7 


2.86 


49 


2.94 


Satisfaction with services 


7 


21.57 


49 


19.59 


Number of services used 


7 


8.71 


49 


8.00 


General Perceptions of BSU 


8 


11.63 


50 


10.6 


Admissions index 


97 


49.32 


55 


50.29 


* After adjusting for the effects of Admissions Index 



adjusted GPA and the No Study Skills control group showing the lowest. These differences 
disappeared for the second semester, however, when holding to a strict statistical standard of a 
probability level of .05. Since the probability level was .07, however, it seemed worthwhile to 
note that differences appeared between the No Study Skills group and the First YES group for the 
second semester, also. 

As shown by Table 4, differences between the groups were found in a yariety of other areas, too. 
Both the First YES and Other GE groups thought that Boise State University had impacted them 
more highly than the No Study Skills group, and both of these groups had used more Boise State 
University services as well. Both First YES and Other GE members were more likely than the No 
Study Skills group to meet as part of a study group, a behavior that has often been found to relate 



to retention. 



Several other areas showed differences, but only between the First YES group and the No Study 
Skills group. Compared to No Study Skills students. First YES students were more likely to 
work with other students on projects, express satisfaction with Boise State University services, 
and generally have better perceptions about their first semester at Boise State. For women only. 
First YES students engaged in more conversations with other students than No Study Skills group 
members. The only area where no differences at all appeared between the groups was in the 
number of conversations held with faculty. 



Table 3 

Re-Enrollment Patterns for First YES, 
Other GE, and Control Groups 







N in Group 
Fall 1995 


% Returning 
Spring 1996 


% Returning 
Fall 1996 




Other GE 


117 


89.74 


59.83 


Total Group 


None 


1,118 


80.50 


52.42 




First YES 


210 


89.52 


57.14 




Other GE 


72 


91.67 


61.11 


Full Time 


None 


918 


24.97 


55.88 




First YES 


188 


90.43 


56.91 




Other GE 


45 . 


86.60 


57.78 


Part time 


None 


200 


60.80 


36.50 




First YES 


22 


81.82 


59.09 




Other GE 


43 


93.02 


55.81 


Male 


None 


512 


78.32 


46.48 




First YES 


95 


85.26 


53.68 




Other GE 


74 


87.84 


62.16 


Female 


None 


606 


82.34 


57.43 




First YES 


115 


93.04 


60.00 




0 
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Table 4 

Means for First YES, Other GE, and No Study Skills Control Groups 





(l)None 


(2:) First YES 


(3) Other GE s; 


-Differences 


Variable 


N 


Mean 


. N 


Mean 


N 


Mean 




First term GPA' 


1114 


2.11 


209 


2.31 


114 


2.62 


All groups 


Second term GPA' 


1054 


2.16 


197 


2.35 


106 


2.24 


None^ 


BSU’s Impact* 


125 


18.05 


144 


16.35 


22 


15.19 


1&2, 1&3 


Faculty Conversations 


116 


3.19 


141 


3.33 


22 


3.87 


None 


Student Conversations - Males 


41 


4.88 


58 


4.62 


7 


4.29 


None 


Student Conversations - 
Females 


76 


4.78 


84 


4.96 


15 


4.87 


1&2 


Project Work 


117 


2.66 


142 


3.11 


22 


3.09 


1&2 


Study Group^ 


117 


2.23 


142 


3.11 


22 


3.09 


1&2, 1&3 


Satisfaction w/services 


116 


15.55 


142 


19.16 


22 


17.73 


1&2 


Number of Services Used 


117 


6.81 


142 


8.28 


22 


8.86 


1&2, 1&3 


General Perceptions of BSU 


125 


11.4 


145 


10.50 


22 


9.91 


1&2 


Admissions Index 


854 


48.9 


180 


46.14 


72 


33.78 


N/A 


1st Term Credits attempted 


1168 


10.86 


210 


11.35 


117 


8.63 


N/A 


Age 


1168 


19.80 


210 


19.13 


117 


22.3 


N/A 


' After adjustment for effects of admissions index 

^p = .07, and follow-up showed significant differences between group 1 & 2 (p = .02) 
^ After adjustment for the effects of credits attempted 



Conclusions 

Fall 1995 retention efforts have shown early returns. For the Cluster program, retention was 
higher for participants than for controls, even though differences were not found for a series of 
variables that might have been expected to moderate re-enrollment. For the First YES course, a 
wider variety of differences was found ranging from re-enrollment to grade point average to 
satisfaction with the institution in a variety of ways. It should be noted, however, that for a 
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number of outcomes, there were no differences between students enrolled in First YES and _ 
students enrolled in one of the other GE courses, mainly GE 108. Since the Other GE group of 
students showed lower admissions index scores and lower average credits attempted, the advising 
process may be correctly diverting students into the course which best suits them. Thus, a “one- 
size-fits-all” approach to course offerings should probably be avoided in the future. 

These are beneficial findings that should help us further tailor our retention efforts. A final study 
of Fall 1995 freshmen will look at what factors predict retention for freshmen including academic 
preparation, psychological readiness for college, orientation, service use, social interactions, and 
other experiences that may impact re-enrollment decisions, should round out the findings. 
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APPENDIX A 



APPENDIX A 

End-of-Term New Student Survey 
Fall, 1995 



Directions; This survey has been designed to gain information about your experiences at BSU this term. Though we have asked for 
your social security number, no information about you as an individual will be reported to anyone. We simply need the information to 
match it to the survey you took at the beginning of the term and to check college records to see if you re-enroll next term. This 
information will be combined to produce a report on the entire group to help answer questions about how to better retain students. If 
you have questions about this process, please talk to your instructor or contact Marcia Belcher, Coordinator of Institutional 
Assessment, at 385-1 1 17. 



1 


Social Security Number; -• ' 
Lart' Name ’ 


'r ' — 


First Name " 










The following are a list of abilities or skills one might expect to develop while pursuing a post-secondary 
education. Please indicate the impact BSU has had on your development in each of these areas: 



Major Impact 

Moderate Impact 

Minor Impact 
I No Impact 

V V 







® 




1. 


Making and exercising a lifelong commitment to learning. 






® 




2. 


Defining and solving problems. 


® 


® 


® 


® 


3. 


Developing skills tliat employers need. 


® 


® 




® 


4. 


Learning about existing and emerging career options. 


® 


® 


® 


® 


5. 


Getting along with people from various cultures, races, backgrounds, etc. 


® 


® 


® 


® 


6. 


Recognizing and using eflective oral communication skills. 


® 


® 


® 


® 


7. 


Recognizing and using effective written communication skills. 


® 


® 


® 


® 


8. 


Analyzing and drawing conclusions from various types of data. 



Indicate how you feel about each of the following statements. 



Strongly Agree 
Agree 



No Opinion 

Disagree 



^ ^ Strongly Disagree 



® 


® 


® 


® 




9. 


BSU’s environment is warm and friendly. 


® 


® 


® 


® 




10. 


I am convinced I made the right decision in choosing BSU. 


® 


® 


® 


® 




11. 


Faculty members are genuinely interested in the welfare of students. 


® 


® 


® 


® 




12. 


Staff members have been helpful to me. 


® 


® 


® 


® 




13. 


I generally experienced good teaching this semester. 


® 


® 


® 


® 




14. 


I plan to re-enroll next semester. 



(Please turn over) ^ 



For each service listed, indicate whether you have used the seiv ice, and 
if vou used-the seK'ice,your level of satisfaction with that service. 

I Did Not Use This Service 
I used this service and was: 

Very Satisfied 
Satisfied 
Neutral 

Dissatisfied 



V V V V 



Very Dissatisfied 

I y 



® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 



(D @ ® © 
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© © ® © © 

© © ® © © 

© © ® © © 

© © ® © © 

© © ® © © 

© © ® © © 

© © ® © © 

© © ® © © 
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1 5. Course registration processes. 

16. Financial aid services. 

17. Academic advising services. 

1 8. Student employment services. 

1 9. Residence halls and apartments. 

20. Food services. 

2 1 . Student health services. 

22. Day care services. 

23. Honors Program. 

24. Library' facilities and services. 

25. Counseling services. 

26. University-sponsored tutorial services. 

27. Cultural programs and activities. 

28. College-sponsored social activities. 



29. Recreational and intramural programs 
and services. 

30. Campus nevvsletters/nevvspapers. 



How frequently did you experience each of the following 
this semester: 



Never 
Once 



® © 
® © 
® © 
® © 
® © 
® © 



V V 



2-3 times 

1 4-6 times 

1 7 or more times 

I ^ 



® © 


31. 


® © 


32. 


® © 


33. 


® © 


34. 


® © 


35. 


® © 


36. 



faculty member. 

Held a conversation with 
another student. 

Worked with other students 
outside of class on a project. 

Met as a member of a study 
group. 

3 5 . Felt lost or alone on campus. 

Experienced rudeness on 
campus. 



37. What is the most important thing you learned this 














38. What should BSU do differently to help students 
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